Papers with acceleration method
Recurrence Boosts Diversity! Revisiting Recurrent Latent Variable in Transformer-Based Variational AutoEncoder for Diverse Text Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Variational Auto-Encoder (VAE) has been widely adopted in text generation due to its ability to learn flexible representations. |
| Approach: | They propose a Transformer-based recurrent VAE structure that imposes recurrence on segment-wise latent variables with arbitrarily separated text segments and constructs the posterior distribution with residual parameterization. |
| Outcome: | The proposed structure can deduce a non-zero lower bound of the KL term and enhance the entanglement of each segment and preceding latent variables, providing a theoretical guarantee of generation diversity. |
Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules (2023.findings-emnlp)
Copied to clipboard
Chaojun Xiao, Yuqi Luo, Wenbin Zhang, Pengle Zhang, Xu Han, Yankai Lin, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou
| Challenge: | Large language models (LLMs) have been successful on NLP tasks but require huge parameter sizes and computational resources. |
| Approach: | They propose a parameter-efficient acceleration method that enhances computational efficiency through plug-and-play compression plugins. |
| Outcome: | The proposed method saves 53% computational costs using only 0.9% additional parameters with a performance drop of less than 2%. |